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Abstract 

Experimental results show that certain message passing algorithms, namely, Survey Propa- 
gation, are very effective in finding satisfying assignments for random satisfiable 3CNF formulas 
which are considered hard for other SAT heuristics. Unfortunately, rigorous understanding of 
this phenomena is still lacking. In this paper we make a modest step towards providing rigorous 
explanation for the effectiveness of message passing algorithms. We analyze the performance of 
Warning Propagation, a popular message passing algorithm that is simpler than Survey Propaga- 
tion. We show that for 3CNF formulas drawn from a certain distribution over random satisfiable 
3CNF formulas, commonly referred to as the planted-assignment distribution, running Warning 
Propagation in the standard way (run message passing until convergence, simplify the formula 
according to the resulting assignment, and satisfy the remaining subformula, if necessary, using 
a simple "off the shelf heuristic) works when the clause-variable ratio is a sufficiently large 
constant. We are not aware of previous rigorous analysis of message passing algorithms for 
satisfiability instances, though such analysis was performed for decoding of Low Density Parity 
Check (LDPC) Codes. We discuss some of the differences between results for the LDPC setting 
and our results. 
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1 Introduction 



A CNF formula over the variables x\, X2, • x n is a conjunction of clauses C±, C2, C m where each 
clause is a disjunction of one or more literals. Each literal is either a variable or its negation. A 
formula is said to be in /c-CNF form if every clause contains exactly k literals. A CNF formula is 
satisfiable if there is a boolean assignment to the variables such that every clause contains at least 
one literal which evaluates to true. 3SAT, the language of all satisfiable 3CNF formulas, is well 
known to be NP-complete [TT]. Hastad [20] proves that it is NP-hard to approximate MAX-3SAT 
(the problem of finding an assignment that satisfies as many clauses as possible) within a ratio 
better than 7/8 (which is the expected number of clauses satisfied by a random assignment). 

The plethora of worst-case NP-hardness results for many interesting optimization problems 
motivates the study of heuristics that give "useful" answers for "typical" subset of the problem 
instances, where "useful" and "typical" are usually not well defined. One way of evaluating and 
comparing heuristics is by running them on a collection of input instances ("benchmarks"), and 
checking which heuristic usually gives better results. Though empirical results are sometimes 
informative, we seek more rigorous measures of evaluating heuristics. One possibility of rigourously 
modeling such "average" instances is to use random models. 

In this paper we analyze the performance of Warning Propagation (WP for brevity), a popular 
message passing algorithm, when applied to satisfiable formulas drawn from a certain random dis- 
tribution over satisfiable 3CNF formulas, commonly called the planted distribution. We show that 
the standard way of running message passing algorithms - run message passing until convergence, 
simplify the formula according to the resulting assignment, and satisfy the remaining subformula, if 
possible, using a simple "off the shelf heuristic - works for planted random satisfiable formulas with 
a sufficiently large yet constant clause-variable ratio. The effectiveness of message passing algo- 
rithms was experimentally shown for "hard" formulas [8], for which other heuristics fail; however no 
rigorous analysis backs up these results. Our result is the first to rigorously prove the effectiveness 
of a message passing algorithm for the solution of a non-trivial random SAT distribution. 

1.1 Average case analysis 

Algorithmic theory of random structures has been the focus of extensive research in recent years 
(see [H] for a survey). As part of this trend, uniformly random 3CNFs (generated by select- 
ing at random m = m(n) clauses over the variables {x±, x n }) have attracted much attention. 
Random 3CNFs are known to have a sharp satisfiability threshold in the clause- variable ratio [17] . 
Namely, a random 3CNF with clause-variable ratio below the threshold is satisfiable whp (with 
high probability, meaning with probability tending to 1 as n goes to infinity ) and one with ratio 
above the threshold is unsatisfiable whp. This threshold is not known exactly (and not even known 
to be independent of n). The threshold is known to be at least 3.52 [23] and at most 4.506 |13j . 
Experimental results indicate that the threshold is closer to the higher end of the interval [12] . 

In this work we mainly consider formulas with large clause- variable ratio. At such ratios almost 
all 3CNF formulas are not satisfiable, therefore more refined definitions are due. We consider 
the planted distribution, denoted throughout by "Pnjp nt - A random 3CNF in this distribution 
is obtained by first picking an assignment ip to the variables, and then including every clause 
satisfied by ip with probability p = p(n), thus guaranteeing that the resulting instance is satisfiable. 
Throughout, we use ip to denote the planted assignment when the relevant instance is clear from 
context. 
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Planted-solution distributions are favored by many researchers, e.g. |16 1 [6|[2^ 1 l25| in the context 
of 3SAT, and also for other graph problems such as the of planted clique, planted bisection, planted 
coloring, and planted bipartite hypergraphs studied e.g. in [3"1 HI \7\ l22l PLI] . 

2 Warning Propagation 

Warning Propagation (WP) is a simple iterative message passing algorithm, and serves as an excellent 
intuitive introduction to more involved message passing algorithms such as Belief Propagation [?] 
and Survey Propagation [8]. These algorithms are based on the cavity method in which the messages 
that a clause (or a variable) receives are meant to reflect a situation in which a "cavity" is formed, 
namely, the receiving clause (or variable) is no longer part of the formula. Messages in the WP 
algorithm can be interpreted as "warnings", telling a clause the values that variables will have if 
the clause "keeps quite" and does not announce its wishes, and telling a variable which clauses will 
not be satisfied if the variable does not commit to satisfying them. We now present the algorithm 
in a formal way. 

Let T be a CNF formula. For a variable x, let N + (x) be the set of clauses in T in which x appears 
positively (namely, as the literal x), and N~(x) be the set of clauses in which x appears negatively. 
For a clause C, let iV + (C) be the set of variables that appear positively in C, and respectively 
N~ (C) for negative ones. There are two types of messages involved in the WP algorithm. Messages 
sent from a variable x\ to a clause Cj in which it appears: 

Xi ► Cj = 'y Ck ► Xi — y ^ Ck ► Xi. 

If Xi appears only in Cj then we set the message to 0. The intuitive interpretation of this message 
should be x\ signals Cj what is currently its favorable assignment by the other clauses it appears in (a 
positive message means TRUE, negative one means FALSE and a message means UNASSIGNED). 
The second type are messages sent from a clause Cj to a variable X{ appearing in Cj\ 

Cj ->■ Xi = I<o(xk -> Cj) I>o(x k ->■ Cj), 

x k £N+(Cj),k^i x k eN-(Cj),k^i 

where /<o(fr) is an indicator function which is T' iff b < (respectively i>o)- If Cj contains only 
Xi (which cannot be the case in 3CNF formulas) then the message is set to 1. Cj — > X{ = 1 can 
be intuitively interpreted as Cj sending a warning to X{ asking it to commit to satisfying Cj (as 
all other literals signaled Cj that currently they evaluate to FALSE). Lastly, we define the current 
assignment of a variable Xi to be 

Bi = y ^ Cj ► Xi — y ^ Cj ► Xi. 

Cj£N+(xi) C 3 &N-{xi) 

If Bi > then x is assigned TRUE, if Bi < then x^ is assigned FALSE, otherwise Xi is UNAS- 
SIGNED. Assume some order on the clause-variable messages (e.g. the lexicographical order 
on pairs of the form (j,i) representing the message Cj — ► Xj). Given a vector a E {0, l} 3m 
in which every entry is the value of the corresponding Cj — > Xi message, a partial assignment 
ip € {TRUE, FALSE, UN ASSIGNED}™ can be generated according to the corresponding Bi 
values (as previously explained). 
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2.1 3SAT and Factor Graphs 



Given a 3CNF formula T on n variables and m clauses, the factor graph (e.g. [?]) of denoted 
by FG(J-), is the following graph representation of T . The factor graph is a bipartite graph, 
FG(F) = {V\ U V2,E) where V\ = {xi,X2, ■■■,x n } (the set of variables) and V2 = {Ci,C2, ...,C m } 
(the set of clauses), (xi, Cj) G E iff X{ appears in Cj. For a 3CNF T with m clauses it holds that 
j^E = 3m, because every clause contains exactly 3 different variables. (Here and elsewhere, j^A 
denotes the cardinality of a set A. The notation \a\ will denote the absolute value of a real number 
a.) 

It would be convenient to think of the messages in terms of the corresponding factor graph. 
Every undirected edge (xi,Cj) of the factor graph is replaced with 2 anti-parallel directed edges, 
(xj — ► Cj) associated with the message x$ — ► Cj and respectively the edge (Cj — > Xj). 

2.2 The Warning Propagation Algorithm 

Warning Propagation (CNF formula J 7 ) : 

1. construct the corresponding factor graph FG(J-). 

2. randomly initialize the clause-variable messages to or 1. 

3. repeat until no clause-variable message changed from the 
previous iteration: 

3. a randomly order the edges of FG(J-). 

3.b update all clause-variable messages Cj — ► xi according 
to the random edge order. 

4. compute a partial assignment tp according to the B{ messages. 

5 . return ip. 

In the above description it might seem as it no update of variable-clause messages is carried 
out. However, these updates are implicit in line 3.b. Namely, when evaluating the clause-variable 
message along the edge C — ► x, C = (x V y V z), the variable-clause messages concerning this 
calculation (z, y — > C) are evaluated on-the-fly using the last updated values Cj — ► y, Cj — ► z 
(allowing feedback from the same iteration). We allow the algorithm not to terminate (the clause- 
variable messages may keep changing every iteration). If the algorithm does return an assignment 
ip then we say that it converged. In practice it is common to limit in advance the number of 
iterations, and if the algorithm does not converge by then, return a failure. 

3 Related Work 

Currently, the Survey Propagation [8] algorithm experimentally outperforms all known algorithms 
in finding satisfying assignments to uniformly random 3CNF formulas with clause-variable ratio p 
close to the satisfiability threshold (4 < p < 4.25). However, theoretical understanding of Survey 
Propagation and other message passing algorithm for random SAT problems is still lacking. This 
should be compared with the success of message passing algorithms for decoding low-density-parity- 
check (LDPC) codes [19] . Here, the experimental success of message passing algorithms (19] was 
recently complemented rigourously by a large body of theoretical work, see e.g. [261 EH EZ] • Some 
important insights emerge from this theoretical work. In particular, it is shown that the quality 
of decoding improves exponentially with the number of iterations - thus all but a small constant 



4 



fraction of the received codeword can be decoded correctly using a constant number of iterations. 
Our analysis of WP on Vn]p nt shows that much of the coding picture is valid also for V%]p nt thus 
providing important insights as to the success of message passing algorithms for random satisfiability 
problems. The planted 3SAT model is similar to LDPC in many ways. Both constructions are based 
on random factor graphs. In codes, the received corrupted codeword provides noisy information 
on a single bit or on the parity of a small number of bits of the original codeword. In Vn]p Ut , f 
being the planted assignment, the clauses containing a variable Xi contain noisy information on the 
polarity of <p(xi) in the following sense - each clause contains Xi in a polarity coinciding with <p(xi) 
with probability 4/7. The SAT setting is however more involved than its coding counterpart; for 
example a SAT instance may have many satisfying assignments (which is whp the case in VK]p nt 
with clause- variable ratio of order o(logn)) whereas a transmitted codeword has a unique true 
solution. More discussion follows in Section [H 

As for relevant results in random graph theory, the seminal work of Alon and Kahale [3] paved 
the road towards dealing with large-constant-degree planted distributions. [3] present an algorithm 
that whp fc-colors planted /s-colorable graphs (the distribution of graphs generated by partitioning 
the n vertices into k equally-sized color classes, and including every edge connecting two different 
color classes with probability p; commonly denoted G n p k) with a sufficiently large constant ex- 
pected degree. Building upon the techniques introduced in [3], Chen and Frieze [22] present an 
algorithm that 2-colors large constant degree planted 3- uniform bipartite hypergraphs, and Flax- 
man |16) presents an algorithm for satisfying large-constant clause-variable ratio planted 3SAT 
instances. 

Though in our analysis we use similar techniques to the aforementioned works, our result is 
conceptually different in the following sense. In [31 l22"l [T6] the starting point is the planted distri- 
bution, and then one designs an algorithm that works well under this distribution. The algorithm 
may be designed in such a way that makes its analysis easier. In contrast, our starting point is 
a given message passing algorithm (WP), and then we ask for which input distributions it works 
well. We cannot change the algorithm in ways that would simplify the analysis. This is similar 
in spirit to the work of [2] who showed that RWalkSat works on very sparse uniformly random 
3CNF instances (for which other simple heuristics are also known to work) , and to the work in jT5] , 
where a certain version of the fc-opt heuristic is shown to work on 'Pnjp nt - Another kind of inter- 
play between algorithms and random distributions is involved in the work on the lower end of the 
satisfiability threshold. Much of it is based on the analysis of simple heuristics, often too simple to 
be of practical value (e.g., in [9] the pure-literal heuristic is used for very sparse uniformly random 
3CNF instances). 

Another difference between our work and that of [3j l22l [l~6] is that unlike the algorithms analyzed 
in those other papers, WP is a randomized algorithm, a fact which makes its analysis more difficult. 
We could have simplified our analysis had we changed WP to be deterministic (for example, by 
initializing all clause- variable messages to 1 in step 2 of the algorithm) , but there are good reasons 
why WP is randomized. For example, it can be shown that (the randomized version) WP converges 
with probability 1 on 2CNF formulas that form one cycle of implications, but might not converge 
if step 4 does not introduce fresh randomness in every iteration of the algorithm (details omitted). 
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4 Our Results 



Given a 3CNF J 7 , simplify T according to ip, when ip is a partial assignment, means: in every 
clause substitute every assigned variable with the value given to it by t/j. If a clause contains a 
literal which evaluates to true, remove the clause. From the remaining clauses, remove all literals 
which evaluate to false. The resulting instance is not necessarily in 3CNF form, as clauses may 
have any number of literals between and 3. Denote by J-\^ the 3CNF T simplified according 
to ip. Note that may contain empty clauses, in which case it is not satisfiable. For a set of 
variables AQV, denote by J-[A] the set of clauses in which all variables belong to A. 

To better understand our results it would be convenient to have the somewhat informal notion 
of a simple formula in mind. We call a 3CNF formula simple, if it can be satisfied using simple well- 
known heuristics (examples include very sparse random 3CNF formulas which are solvable whp 
using the pure-literal heuristic [9] , formulas with small weight terminators - to use the terminology 
of [2] - solvable whp using RWalkSat, etc). 

Theorem 1. Let T be a 3 CNF formula randomly sampled according to Vn]p nt , where p > d/n 2 , d 
a sufficiently large constant. Then the following holds whp (the probability taken over the choice 
of T , and the random choices in lines 2 and 4 of the WP algorithm). There exists a satisfying 
assignment <p* (not necessarily the planted one) such that: 

(a) WP(T) converges after at most O(logn) iterations. 

(b) Let ip be the partial assignment returned by WP^), let Va denote the variables assigned 
to either TRUE or FALSE in ip, and Vjj the variables left UNASSIGNED. Then for every 
variable x £ Va, ip(x) = <p*(x). Moreover, #Va > (1 — e~®^)n. 

(c) J-\^ is a simple formula which can be satisfied in time 0(n). 

Remark 2. Theorem Q] relates to the planted 3SAT model, but as recent results show [?], it also 
applies to the random 3SAT distribution, in which first a random 3CNF is generated by selecting 
every clause with probability p = p(n), independently of the others, and then conditioning on 
satisfiability (or selecting m = m{n) clauses uniformly at random and conditioning on satisfiability). 
Details omitted. 

Proposition 3. Let J 7 be a 3CNF formula randomly sampled according to Vn]p nt , where p > 
clogn/n 2 , with c a sufficiently large constant, and let ip be its planted assignment. Then whp after 
at most 2 iterations, l/l/P(^ r ) converges, and the returned ip equals ip. 

It is worth noting that formulas in V%]p nt , with n 2 p some large constant, are not known to be 
simple (in the sense that we alluded to above). For example, it is shown in [2] that RWalkSat is 
very unlikely to hit a satisfying assignment in polynomial time when running on a random Vn]p nt 
instance in the setting of Theorem [U 

Comparing our results with the coding setting, the effectiveness of message passing algorithms 
for amplifying local information in order to decode codes close to channel capacity was recently 
established in a number of papers, e.g. [26} 129] . Our results are similar in flavor, however the 
combinatorial analysis provided here allows to recover an assignment satisfying all clauses, whereas 
in the random LDPC codes setting, message passing allows to recover only 1 — o(l) fraction of 
the codeword correctly. In [27] it is shown that for the erasure channel, all bits may be recovered 
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correctly using a message passing algorithm, however in this case the LDPC code is designed so 
that message passing works for it. We on the other hand take a well known SAT distribution and 
analyze the performance of a message passing algorithm on it, without changing either of them to 
ease-up the analysis. Moreover, the SAT setting is more involved, as there are many assignments 
satisfying the formula, while for the erasure channel there is a unique codeword satisfying the 
combinatorial constraints given by the message. 

The remainder of the paper is structured as follows. Section provides an overview that may 
help the reader follow the more technical parts of the proofs. In Section[6]we discuss some properties 
that a typical instance in Vn]p Ut possesses. Using these properties, we prove in Section [7] Theorem Q] 
and Proposition [3l In Section [8] we summarize our results and discuss potentially interesting lines 
for further research. 

5 An overview 

Let us first consider some possible fixed points of the Warning Propagation (WP) algorithm. The 
trivial fixed point is the one in which all messages are 0. One may verify that this is the unique 
fixed point in some cases when the underlying 3CNF formula is very easy to satisfy, such as when 
all variables appear only positively, or when every clause contains at least two variables that do not 
appear in any other clause. A local maximum fixed point is one that corresponds to a strict local 
maximum of the underlying MAX-3SAT instance, namely to an assignment r to the variables in 
which flipping the truth assignment of any single variable causes the number of satisfied clauses to 
strictly decrease. The reader may verify that if every clause C sends a 1 message to a variable if 
no other variable satisfies C under r, and a message otherwise, then this is indeed a fixed point 
of the WP algorithm. Needless to say, the WP algorithm may have other fixed points, and might 
not converge to a fixed point at all. 

Recall the definition of Vn]p Ut - First a truth assignment ip to the variables V = {x\,X2, ...,x n } 
is picked uniformly at random. Next, every clause satisfied by tp is included in the formula with 
probability p (in our case p > d/n 2 , d a sufficiently large constant). There are (2 3 — 1) • (3) clauses 
satisfied by <p, hence the expected size of T is p ■ 7 ■ (3) = 7dn/6 + o(n) (when d is constant, then 
this is linear in n, and therefore such instances are sometimes referred to as sparse 3CNF formulas). 
To simplify the presentation, we assume w.l.o.g. (due to symmetry) that the planted assignment 
ip is the all-one vector. 

To aid intuition, we list some (incorrect) assumptions and analyze the performance of WP on 
a Vn]p Ut instance under these assumptions. 

(a) In expectation, a variable appears in 4(™)p = 2d + o(l) clauses positively, and in 3d/ 2 + o(l) 
clauses negatively. Our first assumption is that for every variable, its number of positive and 
negative appearances is equal to these expectations. 

(b) We say that a variable supports a clause with respect to the planted assignment (which was 
assumed without loss of generality to be the all 1 assignment) if it appears positively in the 
clause, and the other variables in the clause appear negatively. Hence the variable is the only 
one to satisfy the clause under the planted assignment. For every variable in expectation there 
are roughly d/2 clauses that it supports. Our second assumption is that for every variable, 
the number of clauses that it supports is equal to this expectation. 
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(c) Recall that in the initialization of the WP algorithm, every clause- variable message C — > x is 
1 w.p. ^, and otherwise. Our third assumption is that with respect to every variable, half 
the messages that it receives from clauses in which it is positive are initialized to 1, and half 
the messages that it receives from clauses in which it is negative are initialized to 1. 

(d) Recall that in step 3b of WP, clause- variable messages are updated in a random order. Our 
fourth assumption is that in each iteration of step 3, the updates are based on the values 
of the other messages from the previous iteration, rather than on the last updated values of 
the messages (that may correspond either to the previous iteration or the current iteration, 
depending on the order in which clause- variable messages are visited). Put differently, we 
assume that in step 3b all clause-variable messages are evaluated in parallel. 

Observe that under the first two assumptions, the planted assignment is a local maximum of 
the underlying MAX-3SAT instance. We show that under the third and fourth assumption, WP 
converges to the corresponding local maximum fixed point in two iterations. Based on the initial 
messages as in our third assumption, the messages that variables send to clauses are all roughly 
(2d — 3d/2)/2 = d/A. Following the initialization, in the first iteration of step 3 every clause C that 
x supports will send x the message 1, and all other messages will be 0. Here we used our fourth 
assumption. (Without our fourth assumption, WP may run into trouble as follows. The random 
ordering of the edges in step 3 may place for some variable x all messages from clauses in which it 
appears positively before those messages from clauses in which it appears negatively. During the 
iteration, some of the messages from the positive clauses may change from 1 to 0. Without our 
fourth assumption, this may at some point cause x to signal to some clauses a negative rather than 
positive value.) The set of clause- variable messages as above will become a fixed point and repeat 
itself in the second iteration of step 3. (For the second iteration, the fourth assumption is no longer 
needed.) Hence the algorithm will terminate after the second iteration. 

Unfortunately, none of the four assumptions that we made are correct. Let us first see to what 
extent they are violated in the context of Proposition [3l namely, when d is very large, significantly 
above logn. Standard concentration results for independent random variables then imply that the 
first, second and third assumptions simultaneously hold for all variables, up to small error terms 
that do not effect the analysis. Our fourth assumption is of course never true, simply because 
we defined WP differently. This complicates the analysis to some extent and makes the outcome 
depend on the order chosen in the first iteration of step 3a of the algorithm. However, it can be 
shown that for most such orders, the algorithm indeed converges to the fixed point that corresponds 
to the planted assignment. 

The more difficult part of our work is the case when d is constant (though a sufficiently large 
constant), as in the case of Theorem[TJ In this case, already our first two assumptions are incorrect. 
Random fluctuations with respect to expected values will whp cause a linear fraction of the variables 
to appear negatively more often than positively, or not to support any clause (with respect to the 
planted assignment). In particular, the planted assignment would no longer be a local maximum 
with respect to the underlying MAX-3SAT instance. Nevertheless, as is known from previous 
work |16j . a large fraction of the variables will behave sufficiently close to expectation so that the 
planted assignment is a local maximum with respect to these variables. Slightly abusing notation, 
these set of variables are often called the core of the 3CNF formula. Our proof plan is to show that 
WP does converge, and that the partial assignment in step 4 assigns all core variables their correct 
planted value. Moreover, for non-core variables, we wish to show that the partial assignment does 
not make any unrecoverable error - whatever value it assigns to some of them, it is always possible 
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to assign values to those variables that are left unassigned by the partial assignment so that the 
input formula is satisfied. The reason why we can expect such a proof plan to succeed is that it is 
known to work if one obtains an initial partial assignment by means other than WP, as was already 
done in [16j [15] . 

Let us turn now to our third assumption. It too is violated for a linear fraction of the variables, 
but is nearly satisfied for most variables. This fact marks one point of departure for our work 
compared to previous work [W\ I15|. Our definition of the core variables will no longer depend only 
on the input formula, but also on the random choice of initialization messages. This adds some 
technical complexity to our proofs. 

The violation of the fourth assumption is perhaps the technical part in which our work is most 
interesting. It relates to the analysis of WP on factor graphs that contain cycles, which is often a 
stumbling point when one analyzes message passing algorithms. Recall that when d is very large 
(Proposition [3]) , making the fourth assumption simplifies the proof of convergence of WP. Hence 
removing this assumption in that case becomes a nuisance. On the other hand, when d is smaller (as 
in Theorem [I]), removing this assumption becomes a necessity. This will become apparent when we 
analyze convergence of WP on what we call free cycles. If messages in step 3b of WP are updated 
based on the value of other messages in the previous iteration (as in our fourth assumption), then 
the random choice of order in step 3a of WP does not matter, and one can design examples in 
which the messages in a free cycle never converge. In contrast, if messages in step 3b of WP are 
updated based on the latest value of other messages (either from the previous iteration or from the 
current iteration, whichever one is applicable), free cycles converge with probability 1 (as we shall 
later show). 

To complete the proof plan, we still need to show that simplifying the input formula according 
to the partial assignment returned by WP results in a formula that is satisfiable, and moreover, 
that a satisfying assignment for this sub-formula can easily be found. The existential part (the 
sub-formula being satisfiable) will follow from a careful analysis of the partial assignment returned 
by WP. The algorithmic part (easily finding an assignment that satisfies the sub-formula) is based 
on the same principles used in [U [16] , showing that the sub-formula breaks into small connected 
components. 

6 Properties of a Random P^T 1 * Instance 

In this section we discuss relevant properties of a random 7 3 njp nt instance. This section is rather 
technical in nature. The proofs are based on probabilistic arguments that are standard in our 
context. Thus we sometimes present only an outline of a proof, when the details can be easily 
completed by the reader. In cases where our argument is more tricky we give the complete proof 
(most notably, Proposition [T3j) . 

In the rest of the paper, for simplicity of presentation, we assume w.l.o.g. that the planted 
assignment is the all TRUE assignment. 

6.1 Stable Variables 

Definition 4. A variable x supports a clause C with respect to a partial assignment tp, if it is 
the only variable to satisfy C under ip, and the other two variables are assigned by ip. 
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Proposition 5. Let T be a 3 CNF formula randomly sampled according to Vn,p nt , where p > d/n 2 , 
d a sufficiently large constant. Let Fsupp be a random variable counting the number of variables 
in T whose support w.r.t. ip is less than d/3. Then whp Fgjjpp < e~®( d ^n. 

Proof. (Outline) The proposition follows from simple concentration arguments. Every variable is 
expected to support • (™) = f + O(^) clauses, thus using e.g. Chernoff 's bound and linearity of 
expectation, one obtains E[F SU pp] < e~ e ^n. To prove concentration around the expected value 
one can use the Chernoff bound once more as the support of one variable is independent of the 
others (since it concerns different clauses which are included independently of each other). ■ 

Following the definitions in Section [21 given a CNF T and a variable x, we let N ++ (x) be the 
set of clauses in T in which x appears positively but doesn't support w.r.t. ip. Let N s (x) be the 
set of clause in T which x supports w.r.t. (p. Let ir = it{J-) be some ordering of the clause-variable 
message edg 6S in the factor graph of J~ . For an index i and. a literal £ x (by £x 

we denote a literal 

over the variable x) let ^~ l {i x ) be the set of clause- variable edges (C — ► x) that appear before index 
i in the order tt and in which x appears in C as £ x . For a set of clause- variable edges £ and a set 
of clauses C we denote by £ n C the subset of edges containing a clause from C as one endpoint. 

Definition 6. A variable x is stable in T w.r.t. an edge order tt if the following holds for every 
clause-variable edge C — > x (w.l.o.g. assume C = [l x V i y V £ z ), C — > x is the i 'th message in tt): 

(a) |#7r-*(y) n N++{y) - #^(y) n N~(y)\ < d/30. 

(b) \#N++(y)-#N-(y)\ < d/30. 

(c) #iV%) > d/3 
and the same holds for z. 

Proposition 7. Let J 7 be a 3 CNF formula randomly sampled according to Vn]p nt , where p > d/n 2 , 
d a sufficiently large constant. Let tt be a random ordering of the clause- variable messages, and 
Fun stab be a random variable counting the number of variables in T which are not stable. Then 
whp F UNS tab < e" e(d) n. 

Proof. We start by bounding E[Fjjnstab]- Consider a clause- variable message edge C — > x in 
location i in tt, C = (£ x V i y V £ z ). Now consider location j < i. The probability of an edge C — > y 
in location j is (3(g)) / (7(g)) = ^ + O(^) which is exactly the probability of an edge C" — > y, 
C" G N ++ (y). This implies 

EMn-^y) n N ++ {y) - #vr^(y) n N~(y)\] = 0. 

If however 

|#7r%) n N+ + (y) - #vr- 4 (y) n N~(y)\ > d/30 

then at least one of the quantities deviates from its expectation by d/60. 

Look at ifTT~ l (y) D N ++ (y) - this is the number of success in draws without replacement. It is 
known that this quantity is more concentrated than the corresponding quantity if the draws were 
made with replacement |21j . In particular, since the expectation of #vr _J (y) n N ++ (y) is 0{d) it 
follows from Chernoff's bound that the probability that it deviates from its expectation by more 
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than d/60 is e A similar statement holds for ffix l (y) n N (y). Properties (b) and (c) are 

bounded similarly using concentration results. 

The calculations above hold in particular for the first 5d appearances of messages involving x. As 
for message 5d + 1, the probability of this message causing x to become unstable is bounded by 
the event that x appears in more than 5d clauses. As x is expected to appear in 3.5d clauses, the 
latter event happens w.p. e -0 ^ (again using standard concentration results). To sum up, 

Pr[x is unstable ] < 5d ■ e~ e{d) + e~ &W = e~ e{d) . 

The bound on E[Funstab] follows by linearity of expectation. 

We are now left with proving that Fun stab is concentrated around its expectation, we do so 
using a martingale argument. Define two new random variables, F\ counting the number of unstable 
variables x s.t. there exists a clause C, containing x, and another variable y, s.t. y appears in more 
than logn clauses, and F2 to be the unstable variables s.t. in all clauses in which they appear, all 
the other variables appear in at most logn clauses. Observe that Fun stab = Fi + F%. To bound 
F±, observe that if Fi > 1, then in particular this implies that there exists a variable which appears 
in more than logn clauses in T. This however can be shown not to happen whp (since every 
variable is expected to appear only in 0(d) clauses). To bound F2 we use a martingale argument 
in the constellation of [5], page 101. We use the clause-exposure martingale (the clause-exposure 
martingale implicitly includes the random ordering ir, since one can think of the following way 
to generate the random instance - first randomly shuffle all possible clauses, and then toss the 
coins). The exposure of a new clause C can change F2 by at most 6 logn since every variable in 
C appears in at most log n clauses, namely with at most 2 log n other variables that might become 
(un)stable due to the new clause. The martingale's total variance, to use the terminology in [5], is 
a 2 = @(dnlog 2 n). Using inequality (7.1) in [5] page 101, with a = e -0 ^ yW log n, and the fact 
that E[F2] < E[Funstab], concentration around the expectation of F2 is obtained. ■ 

Let a G {0, l} 3 *-^ be a clause-variable message vector. For a set of clause-variable message 
edges £ let l a (£) be the set of edges along which the value is 1 according to a. For a set of clauses 
C, l a (C) denotes the set of clause-variable message edges in the factor graph of T containing a 
clause from C as one endpoint and along which the value is 1 in a. 

Definition 8. A variable x is violated by a inn if there exists a message C — > x, C = [l x \/£yVl z ), 
in place i in tt s.t. one of the following holds: 

(a) |#i a (vr-%) n N++(y)) - #1^^) fl N~{y))\ > d/30 

(b) \#l a (N++(y)) - #l a (N-(y))\ > d/30 

(c) #l a (N s (y)) < d/7. 

Or one of the above holds for z. 

Proposition 9. Let T be as in the setting of Theorem d, and let X be a set of stable variables 
w.r.t. an arbitrary ordering tt. Let a be a random clause-variable message vector. Let Fyio be a 
random variable counting the number of violated variables in X. Then, Fyio < e~ ^^X. 

Proof. As in the proof of Proposition [71 we first bound E[Fyjo], and then prove concentration 
using a martingale argument. Since the martingale argument is the same as Proposition [7J (instead 
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of a clause-exposure martingale, we have a clause- variable message values exposure martingale), we 
just show how to bound the expectation. 

Consider a stable variable x in T w.r.t. to an ordering ir of the clause-variable messages. Let a 
be a random assignment to the clause-variable messages. Consider a clause-variable message edge 
C — > x at location i in ir. x is stable and therefore 

|#7r-*(y) n N ++ (y) - #7r- i (y) n N~(y)\ < d/30. 

Since a is a random assignment 

E[\#l a (7T- l (y) n N ++ (y)) - #l a (^(y) n iV~(y))|] < d/60. 

If however 

\#l a (7r-\y) n N ++ (y)) - #l a (vr- l (y) n N~(y))\ > d/30, (6.1) 

then at least one of the quantities in (|6.ip deviated from its expectation by at least (d/30 — d/60)/2. 
Since both quantities are binomially distributed with expectation 0(d), the probability of the latter 
happening is e~®( d \ using standard concentration results. Properties (b) and (c) are bounded 
similarly using tight concentration results. Using the union bound as in the proof of Proposition [7] 
and the linearity of expectation the bound on the expectation follows. ■ 



6.2 Dense Subformulas 

The next property we discuss is analogous to a property proved in [3] for random graphs. Loosely 
speaking, [3] prove that whp a random graph doesn't contain a small induced subgraph with a 
large average degree. A similar proposition for 3SAT can also be found in [16j . 

Proposition 10. Let c ^ 1 be an arbitrary constant. Let J 7 € Vn^be as in the setting of Theorem 
d Then whp there exists no subset of variables U, s.t. $U < e~®( d ^n and there are at least c#U 
clauses in T containing two variables from U . 

Proof. (Outline) For a fixed set U of variables, #U = k, the number of clauses containing two 
variables from U is 



Each of these clauses is included independently w.p. \. Thus, the probability that ck of them are 
included is at most 



lk 2 n\ fd_\ ck < Uk 2 ne ^_\ ck < / 12kd\ ck 
ck J \n 2 J ~ \ ck n 2 J ~ \ cn J 

Using the union bound, the probability there exists a "dense" set U is at most 



e 



G(d) n , N /ln , JX ck 



The last equality is obtained using standard calculations. 



2c i2c-2\ 
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6.3 The Core Variables 



We describe a subset of the variables, denoted throughout by Ti and referred to as the core variables, 
which plays a crucial role in the analysis. The notion of a stable variable is not enough to ensure 
that the algorithm will set a stable variable according to the planted assignment, as it may happen 
that a stable variable x appears in many of its clauses with unstable variables. Thus, x can be biased 
in the wrong direction (by wrong we mean disagreeing with the planted assignment). However, if 
most of the clauses in which x appears contain only stable variables, then this is already a sufficient 
condition to ensure that x will be set correctly by the algorithm. The set Ti captures the notion 
of such variables. There are several ways to define a set of variables with these desired properties, 
we present one of them, and give a constructive way of obtaining it (though it has no algorithmic 
implications, at least not in our context). 

Formally, Ti = Ti(J-,<p,a,ir) is constructed using the following iterative procedure: 

Let A\ be the set of variables whose support w.r.t. ip is at most d/3. 

Let A 2 be the set of non-stable variables w.r.t. ir. 

Let A3 be the set of stable variables w.r.t. ir which are violated by a. 

(a) Set H = V\(A 1 UA 2 U A 3 ). 

(b) While there exists a variable ai S Hi which supports less than d/A clauses in J-[Hi] OR appears 
in more than d/30 clauses not in J- [Hi] define Hi + \ = Hi \ {ai}. 

(c) Let a m be the last variable removed in step 2. Define Ti = H m+ \ . 



Proposition 11. // both a and tt are chosen uniformly at random then whp j^-Ti > (1 — e 0( -^)ri. 

Proof. Let Ti = V \ Ti. Set 5 = e~®( d \ Partition the variables in Ti. into variables that belong to 
A\ U A2 U A3, and variables that were removed in the iterative step, H lt = Hq \ Ti. If j^-Ti > Sn, 
then at least one of A\ U A2 U A3, H lt has cardinality at least 5n/2. Consequently, 

Pr[#H > Sn] < Pr[#Ai UA 2 UA 3 > Sn/2] + Pr{#H' lt > 5n/2 I #A 1 U A 2 U A 3 < Sn/2] . 

' v ' > v ' 

(a) (6) 

Propositions El EJ and [9] and Azuma's inequality for example are used to bound (a). To bound (6), 
observe that every variable that is removed in iteration i of the iterative step (step 2), supports at 
least (d/3 — d/A) = d/12 clauses in which at least another variable belongs to {a±, a 2 , fli-i} U A\ U 
A 2 U A3, or appears in d/30 clauses each containing at least one of the latter variables. Consider 
iteration 5n/2. Assuming #A\ U A 2 U A3 < 5n/2, by the end of this iteration there exists a set 
containing at most 5n variables, and there are at least d/30 ■ Sn/2 ■ 1/3 clauses containing at least 
two variables from it (we divide by 3 as every clause might have been counted 3 times). Plugging 
c = c?/ 180 in Proposition [TUl (6) is bounded. ■ 

6.4 The Factor Graph of the Non-Core Variables 

Proposition 1111 implies that for p = clogn/n 2 , c a sufficiently large constant, whp Ti contains 
already all variables. Therefore the following propositions are relevant for the setting of Theorem 
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[T] (namely, p = 0(l/n 2 )). 



Proposition 12. Whp every connected component in the factor graph induced by the non-core 
variables contains O(logn) variables. 

A proposition of similar flavor to Proposition [12] was proven in [16] though with respect to a 
different notion of core. Proposition [12] will not suffice to prove Theorem [T] and we need a further 
characterization of the non-core factor graph, which is not present in any of the aforementioned 
works. 

Proposition 13. Whp every connected component in the factor graph induced by the non-core 
variables contains at most one cycle. 

Proposition 14. The probability of a cycle of length at least k in the factor graph induced by the 
non-core variables is at most e~®( dk ) . 

Corollary 15. Let f = f(n) be an arbitrary growing function ofn (namely, f(n) — > oo as n — > oo). 
Then whp there is no cycle of length f(n) in the non-core factor graph 

Since Propositions [T2"iri4l are all proven using similar arguments, we chose to prove Proposition 
[13] which is not present in any of [3] [16]. We proceed with the proof of Proposition [13] and Corollary 

6.5 Proof of Proposition PT3l 

In order to prove Proposition [13] it suffices to prove that whp there are no two cycles with a simple 
path (maybe of length 0) connecting the two. To this end, we consider all possible constellations 
of such prohibited subgraphs and prove the proposition using a union bound over all of them. 

Every simple 2&-cycle in the factor graph consists of k variables, w.l.o.g. say xi,...,X}~ (all 
different), and k clauses Ci,...,Cfc, s.t. Xj,Xj + i € Cj. The cycle itself consists of 2k edges. 

As for paths, we have 3 different types of paths: paths connecting a clause in one cycle with 
a variable in the other (type 1), paths connecting two clauses (type 2), and paths connecting two 
variables (type 3). Clause- variable paths are always of odd length, and clause-clause, variable- 
variable paths are always of even length. A A;-path P consists of k edges. If it is a clause-variable 
path, it consists of (k — l)/2 clauses and the same number of variables. If it is a variable- variable 
path, it consists of k/2—1 variables and k/2 clauses and symmetrically for the clause-clause path (we 
don't take into account the clauses/variables that participate in the cycle, only the ones belonging 
exclusively to the path). 

Our prohibited graphs consist of two cycles C±,C2 and a simple path P connecting them. We 
call a graph containing exactly two simple cycles and a simple path connecting them a bi- cycle. 
The path P can be of either one of the three types described above. Similarly to the bi-cycle case, 
one can have a cycle C and a chord P in it. We call such a cycle a chord- cycle. For parameters 
i,j, k E [l,n], and t £ {1,2,3}, we denote by B 2 i t 2j,k,t a bi-cycle consisting of a 2i-cycle connected 
by a A;-path of type i to a 2j-cycle. Similarly, we denote by Bn,k,t a chord-cycle consisting of a 
2i-cycle with a /c-path of type t as a chord. 

Our goal is then to prove that whp the graph induced by the non-core variables contains no 
bi-cycles and no chord-cycles. 

For a fixed factor graph H we let Fh C T be a fixed minimal set of clauses inducing H, and 
V{H) be the set of variables in H. In order for a fixed graph H to belong to the factor graph 
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induced by the non-core variables it must be that there exists some Fjj s.t. Fjj C T and that 
V(H) C H (put differently, V(#) n H = 0). 

Let -B = B2i s 2j,k,t ( or = ^2i,k,t if -B is a chord-cycle) be a fixed bi-cycle and -Fb a fixed 
minimal-set of clauses inducing B. We start by bounding Pr[FB Q T and V(B) nW = 0] and then 
use the union bound over all possible bi-cycles (chord-cycles) and inducing minimal sets of clauses. 
As the two events - {Fb C J 7 } and {V(B) n W = 0} - are not independent, the calculations are 
more involved. Loosely speaking, to circumvent the dependency issue, one needs to defuse the 
effect that the event {Fb C J 7 } might have on TC. To this end we introduce a set TC*, defined very 
similarly to TC only "cushioned" in some sense to overcome the dependency issues (the "cushioning" 
depends on Fb)- This is done using similar techniques to [3| [16] . 

We start by defining the new set of core variables TC* (again w.r.t. an ordering ir of the clause- 
variable messages and an initial values vector a). The changes compared to TC are highlighted in 
bold. 

Let B\ be the set of variables whose support w.r.t. ip is at most d/3. 

Let B2 be the set of non-stable variables w.r.t. ir where we redefine the gap in Definition® to be 

(d/30-6) in (a) and (b). 
Let B3 be the set of stable variables w.r.t. tt which are violated by a where we redefine the gap in 

Definition^ to be (d/30-6) in (a) and (6). 
Let J C V(B) be the set of variables appearing in no more than 6 different clauses in Fb 

(a) Set Hq = V\ (Bi UB2UB3U (V(F C ) \ J)). 

(b) While there exists a variable ai G H[ which supports less than d/A clauses in !F[H^[ OR 
appears in more than (d/30-6) clauses not in J-\H'j\, define H' i+1 = H- \ {ai}. 

(c) Let a m be the last variable removed at step 2. Define TC* = H' m+1 . = H' m+1 . 



Propositions [7] and [9] could be easily adjusted to accommodate the 6-gap in the new definition in 
Bi and B3. Therefore Proposition [11] can be safely restated in the context of TC*: 

Proposition 16. If both a and tt are chosen uniformly at random then whp f^TC* > (1 — e _ ®^)n. 

Proposition 17. Let b = #V(B), then the set J defined above satisfies #J > 6/4 

Proof. Observe that if Fb is minimal then #Fb < 6 + 1. This is because in every cycle the number 
of variables equals the number of clauses, and in the worst case, the path contains at most one 
more clause than the number of variables, and the same goes for the chord-cycle. Now suppose in 
contradiction that j£J < 6/4, then there are more than 36/4 variables in V(B), each appearing in 
at least 6 different clauses in Fb- Thus, #-Fb > (6 • 36/4)/3 = 1.56 > b + 1 (we divided by three 

b>3 

as every clause might have been counted 3 times), contradicting #Fb < 6 + 1. ■ 

The following proposition "defuses" the dependency between the event that a bi-cycle (chord-cycle) 
was included in the graph and the fact that it doesn't intersect the core variables. In the following 
proposition we fix an arbitrary tt and a in the definition of TC* , therefore the probability is taken 
only over the randomness in the choice of T. 

Proposition 18. Pr[F B C T and V(B) n TC = 0] < Pr [Fb C T] ■ Pr[J n TC* = 0] 
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To prove Proposition [18] we need the following Lemma. 

Lemma 19. For every bi-cycle (chord-cycle) B and every minimal inducing set Fb, TC*(jF, <p, a, ir) C 
H(FUF B ,(p,a,Tr). 

This lemma clarifies the motivation for defining Ti*. It is not necessarily true that H(F) C 
TL{F U Fb)- For example, a variable which appears in H(F) could disappear from TC(F U Fb) since 
the clauses in Fb make it unstable. Loosely speaking, TL* is cushioned enough to prevent such a 
thing from happening. 

Proof. (Proposition ITS]) 

Pr[Fs C .P and y(P)nH = 0] < Pr[P B C .P and JnH = 0] = Pr[JnH = %\F B C ^]Pr[F B C JF]. 
Therefore, it suffices to prove 

Pr[J n H = 0|Fb C JF] < Pr[J n H* = 0]. 
Pr[JnW* = 0] = ^ Pr[.F = P] > ^ Pr[jF = P] 

Break each set of clauses P into F' = F \ Fb and P" = P n Fb , and the latter equals 

Yl Pr ^ \ p B = F' and TnF B = F") 

F':FT\F B =Q,JnH(F'llF B )=$F":F"CF B 

Since the two sets of clauses, F\Fb, and TC\Fb, are disjoint, and clauses are chosen independently, 
the last expression equals, 

Y Y Pr[T \F B = F'\Pr[T n F B = F"] = 

F':F'nF s =0,JnH(F'UF s )=0 F":F"CF B 

Pr[F\F B = F'] ^ Pr[FnF B = F"] = 

F':F'nF s =0,JnH(F'UF s )=0 F":F"CF B 



Pr[F\F B = F'} 

F>:F'nF B =®,JnH{F'UF B )=(& 

Since (J 7 \Fb) DPb = 0, and clauses are chosen independently, the event {Fb C JF} is independent 
of the event {J- \ Fb = F'}. Therefore, the latter expression can be rewritten as 

Y Pr\T \F B = F'\F b QT\ = Pr[J nH = 0|P B C F]. 

F':FT\F B =$,JnH(F'LIF B )=0 



Proof. (Lemma [T9|) The lemma is proved using induction on i (i being the iteration counter in the 
construction of 7i). For the base case H' (F) C Hq{F U Fb), since every variable in H' (F) appears 
in at most 6 clauses in F B it holds that A^F U F B ) C B^F), i = 2,3. A X {F U P B ) C B X (F) 
holds at any rate as more clauses can only increase the support, and the set J was not even con- 
sidered for Hq. Suppose now that H[{J-) C H.{{T U Pc), and prove the lemma holds for iteration 
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i + 1. If x 6 if- +1 (.F) then £ supports at least ci/3 clauses in which all variables are in H'^J 7 ). 
Since H'^J 7 ) C H^J 7 U Fb), then x supports at least this number of clauses with only variables 
of Hi(J-U Fq). Also, x appears in at most d/30 — 6 clauses with some variable outside of H'^J 7 ), 
again since H'^J 7 ) C Hi(J- U .Fb) and -Fb contains at most 6 clauses containing x, x will appear 
in no more than d/30 clauses each containing some variable not in H^TiJ Fb)- We conclude then 
that x G Hi{T U Fb). ■ 



Corollary 20. Let B = B 2 i^,t be a chord-cycle, and let A = l—#H*/n, then Pv[Fb C T and V(B)n 
TC = 0] < fc) where: 

(a) fc) < (d/n 2 )( l+k / 2 ^ • A^ + 2~ x )/ 4 i3 consists of a 2i-cycle and a variable-variable k-path as 
a chord. 

(b) p(i,k) < (d/n 2 )( 4+fc / 2_1 ) • A^ + 2^ 4 if B consists of 2i- cycle and a clause-clause k-path as a 
chord. 

(c) p(i,k) < (d/n 2 )^ + ~^~^ • A^" 1 " - ^ - ^ 4 if B consists of 2i- cycle and a variable- clause k-path as a 
chord. 

Proof. In (a), we have i + | — 1 variables and i + | clauses. Since the clauses are chosen 
independently, 

Pr[F B C J 7 ] < (d/n 2 ) i+ i 

To bound the event { JnTC* = 0}, observe that Fb is fixed in the context of this event, and there is 
no pre-knowledge whether Fb is included in J 7 or not. Therefore, J can be treated as a fixed set of 
variables, thus the choice of TL* is uniformly distributed over J. Recalling that #J > (i + § - l)/4, 
it follows that 

Pr[J n ?r = 0] < i^Li = (j+ t 1)/4 < A^t" 1 )/ 4 . 

W ((i+|-l)/4j 

The last inequality follows from standard bounds on the binomial coefficients, (a) now follows 
immediately from Proposition In the same manner items b, c are proven (just counting how 
many variables and clauses B contains, depending on the type of its path). ■ 

Corollary 21. Let B = B 2 ^2j,k,t be a bi-cycle, and let A = \ — #Ti*/n, then Pr[Fg C T and V(B)n 
H = 0] < p(i,j, k) where: 

(a) p(i,j,k) < (d/n 2 )( l+J+fc / 2 ) • A^ +:,+ 2 _1 V 4 consists of a 2i, 2 j -cycles and a variable-variable 
k-path. 

(b) p(i,j,k) < (d/n 2 )^ +J+fc / 2 ~ 1 ) • A^ +J,+ 2^ 4 i/ -B consists of 2i,2j- cycles and a clause-clause 
k-path. 

(c) p(i,j,k) < (d/n 2 )^ t+ ^ a - ) • / \( i +i+ _ 2 - )/ 4 consists of 2i,2j- cycles and a variable-clause 
k-path. 
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Corollary 1211 is proven in a similar way to Corollary 1201 

To complete the proof of Proposition 1131 we use the union bound over all possible bi/chord- 
cycles. We present the proof for the bi-cycle case; the proof of the chord-cycle is analogous. First 
consider the case where B is a bi-cycles with a variable-variable path (in which case the path must 
be of even length). Let s = Sjj^ = i + j + ^ — 1 (namely, #V{B) = s and #Fb = s + 1). The 
probability of B is then at most 

t ( 5j "J-fe,M)^(T»)«-«-(^)"" +1 -A..,«/.< 



•' -=1 i i -=1 

>2 t ' J '2 X 



IV 7d 

< 



E ?+ E ^Vs( 4 "'s») 3 -?+» 3 -i = °(i)- 



2 / n 



n f— ' ^2/ n n 

*+J + |<41ogn «+i+|>41ogn 

We now move to the case B is a bi-cycles with a clause-clause path (in which case the path again 
must be of even length). Let s = Sjj.fc = i + j + ^ (namely, #V(B) = s and #-Fb = s — 1). Observe 
that in this case the number of classes in Fb is s — 1, however only for s — 3 clauses one has the 
freedom in choosing the third variable (the two clauses which are the endpoints of the path are 
completely determined once the order of the variables is fixed). The probability is then at most 



t {^-y-^^-i^y 1 -^^ t (7e-^ r .i =0( i) 



Lastly, we need to consider the case B is a bi-cycles with a clause- variable path (in which case the 
path must be of odd length). Let s = Sij ; fe = i+j + (namely, s = #V(B) = #Fb)- Again, one 
clause in Fb is completely determined once the the order of the variables is fixed. The probability 
is then at most 

e (4)-(^-(T»)--^r-AW <s 

E C-f)'-''-"- K (i)'- X "^ E (7e-<i.AV*)..i = (l) 

To sum up, the probability of a bi-cycle in the graph induced by the non-core variables is 3 • o(l) = 
o(l). 



6.6 Outline of Proof of Proposition 1141 

The proof is basically the same as that of Proposition [T3l One defines the same notion of "cush- 
ioned" core TC*, and proceeds similarly. We therefore reprove only the last part - the union bound 
over all possible cycles. 
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First let us bound the number of cycles of length k. There are (?) ways to choose the variables 
inducing the cycle, and k\/2 ways to order them on the cycle. As for the set of clauses that induces 
the cycle, once the cycle is fixed, we have at most (7n) fc ways of choosing the third variable and 
setting the polarity in every clause. In what follows we let An be the number of vertices in the 
non-core factor graph. 

Using the union bound, the probability of a cycle of length at least k in the non-core factor 
graph is at most 

£® ■«■<*.>«• (-)'.*» <£(!*)'.,.,,.(<)'.*/» 

t=k ^ y 7 / v> 1 / t=k v 7 v 7 

choose the cycle cyc l e included but doesn't intersect H* ,Prop. [TBI 

An. 

= ^2(7e-d-V\) t 

t=k 



Assuming that Proposition [16] holds (which is the case whp), then A = e -0 ^ and 7e-d- V e~®( d ) = 
e -Q(d) _ which jg muc h smaller than 1. In particular, the last summation is simply the sum of a 
decreasing geometric series with quotient e~ Q<yd \ which sums up to at most twice the first item, 
which is at most e~ Q<ydk \ 

7 Proof of Theorem [I] and Proposition [3] 

We start by giving an outline of the proof of Theorem [TJ Proposition [3] is derived as an easy 
corollary of that proof. 

Recall that to prove Theorem [1] we need to establish three properties: 

(a) Convergence: the WP algorithm converges to a fixed point. 

(b) Consistency: the partial assignment implied by this fixed point is consistent with some sat- 
isfying assignment. 

(c) Simplicity: after simplifying the input formula by substituting in the values of the assigned 
variables, the remaining subformula is not only satisfiable (this is handled by consistency), 
but also simple. 

We assume that the formula T and the execution of WP are typical in the sense that Propositions 
[TTJ \12\ and [13] hold. First we prove that after one iteration WP sets the core variables 7i correctly 
(Bi agrees with ip in sign) and this assignment does not change in later iterations. The proof of 
this property is rather straightforward from the definition of a core. This establishes convergence 
and consistency for the core variables. From iteration 2 onwards WP is basically running on T in 
which variables belonging to 7i are substituted with their planted assignment. This subformula 
is satisfiable. Moreover, its factor graph contains small (logarithmic size) connected components, 
each containing at most one cycle. This last fact serves a dual purpose. It shows that if the 
WP will eventually converge, the simplicity property will necessarily hold. Moreover, it will assist 
us in proving convergence and consistency for the subformula. Consider a connected component 
composed of a cycle and trees "hanging" on the cycle. Proving convergence on the trees is done 
using a standard inductive argument. The more interesting part is proving convergence on the 
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cycle. The difficulty there is that messages on a cycle may have more than one fixed point to which 
they may possibly converge, which makes it more difficult to prove that they converge at all. Our 
proof starts with a case analysis that identifies those cases that have multiple fixed points. On these 
cases we prove that almost surely random fluctuations caused by step 3. a of the WP algorithm will 
lead to convergence to some fixed point. This is similar in flavor to the fact that a random-walk 
on a line eventually reaches an endpoint of the line (even though one cannot tell a-priori which 
endpoint this will be). Hand- in-hand with establishing convergence for the trees and cycle, we shall 
also prove consistency. 

The set Va of Theorem [T] is composed of all variables from Ti and those variables from the 
non-core factor graph that get assigned. The set Vjj is composed of the UNASSIGNED variables 
from non-core factor graph. We now proceed with the formal proof. 

7.1 Analysis of WP on the core factor graph 

We start by proving that the messages concerning the factor graph induced by the core-variables 
converge to the correct value, and remain the same until the end of the execution. 

We say that a message C — > x, C = {£ x \l £ y V £ z ), is correct if its value is the same as it is when 
y — > C and z — > C are 1 (that is agree in sign with their planted assignment). In other words, 
C — > x is 1 iff C = (x V y V z) (x supports C). 

Proposition 22. If Xi G Ti and all messages C — > Xj, C G T\H\ are correct at the beginning of an 
iteration (line 3 in the WP algorithm), then this invariant is kept by the end of that iteration. 

Proof. By contradiction, let Cq — > x be the first wrongly evaluated message in the iteration. 
W.l.o.g. assume Co = {£ x V ^ V £ z ). Then at least one of y, z sent a wrong message to Co- 

C£N+(y),C^C C'eN-(y),C'^C 

Every message C" — > y, C" G F[Ti] n {N ++ (y) U N~(y)} is (since it was correct at the beginning 
of the iteration and that didn't change until evaluating Co — > x). On the other hand, y G TI and 
therefore it supports at least d/A clauses in J-[TC]. Thus at least (d/4 — 1) messages in the left 
hand sum are '1' (we subtract 1 as y might support Co), y appears in at most d/30 clauses with 
non-core variables (all of which may contribute a wrong '1' message to the right hand sum). All 
in all, y — > Co > {d/A — d/30 — 1) > d/5, which is correct (recall, we assume ip = l n ). The same 
applies for z, contradicting our assumption. ■ 

Proposition 23. If x^ £ Ti and all messages C — > Xi, C G J-[Ti] are correct by the end of a WP 
iteration, then Bi agrees in sign with <p{xi) by the end of that iteration. 

Proposition 1231 follows immediately from the definition of Ti and the message Bi. It suffices to show 
then that after the first iteration all messages C — > x^, C G J-[Ti] are correct. 

Proposition 24. If J 7 is a typical instance in the setting of Theorem^ then after one iteration of 
WP(F), for every variable xi G Ti, every message C — > Xi, C G T\H\ is correct. 

Proof. The proof is by induction on the order of the execution in the first iteration. Consider the 
first message C — ► x, C = {l x V l y V i z ), C G J-'l'Ti], to be evaluated in the first iteration. Now 
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consider the message y — ► C at the time C — > x is evaluated. All messages C" — > y, C € ^[-ff] 
have their initial random value (as C — > x is the first core message to be evaluated). Furthermore, 
y £ 7i, and therefore there are at most d/30 messages of the form C" — > y, C" ^ T\H\. x £ TL hence 
it is stable w.r.t. ir and not violated by the initial clause-variable random messages. Therefore 

y^C> d/7 d/30 d/30 > d/U. 

property (c) in dcfn. \E\ property (6) in dcfn. [8] non-core messages 

The same applies to z, to show that C — > x is correct. Now consider a message C —> x at position 
i, and assume all core messages up to this point were evaluated correctly. Observe that every core 
message C —> y that was evaluated already, if C" G {N ++ (y) U N~(y)} n JFfW] then its value is '0' 
by the induction hypothesis. Since x is not violated by a, property (b) in definition [8] ensures that 
to begin with \#l a (N ++ (y)) — #l a (N~ (y))\ < d/30. y &H, therefore it appears in at most d/30 
non-core messages, all of which could have been already wrongly evaluated, changing the above 
difference by additional d/30. As for the core messages of y which were already evaluated, since 
they were evaluated correctly, property (a) in definition [8] ensures that the above difference changes 
by at most additional d/30. All in all, by the time we evaluate C — > x, 

Yl C'^V- E C" ^y>-3-d/30. 

C'eN++(y),C'^C C"eN~(y),C"^C 

As for messages that y supports, property (c) in definition [8] ensures that their contribution is at 
least d/7 to begin with. Every core message in N s (y) that was evaluated turned to '1', every non- 
core message was already counted in the above difference. Therefore y — > C > d/7— 3 -d/30 > d/25. 
The same applies to z showing that C — ► x is correct. ■ 

To prove Proposition [3l observe that when p = c log n/n 2 , with c a sufficiently large constant, 
Proposition 1111 implies H = V. Combining this with Proposition 1241 Proposition [3] readily follows. 



7.2 The effect of messages that already converged 

It now remains to analyze the behavior of WP on the non-core factor graph, given that the messages 
involving the core factor graph have converged correctly. A key observation is that once the messages 
in the factor graph induced by the core variables converged, we can think of WP as if running on 
the formula resulting from replacing every core variable with its planted assignment and simplifying 
(which may result in a 1-2-3CNF). The observation is made formal by the following proposition: 

Proposition 25. Consider a run of WP that has converged on the core. Starting at some iteration 
after WP has converged on the core, consider two alternative continuations of the warning propa- 
gation algorithm. WP\ denotes continuing with WP on the original input formula. WP2 denotes 
continuing with WP on the formula obtained by replacing each core variable with its planted assign- 
ment and simplifying. Then for every iteration t, the sequence of messages in the t 'th iteration of 
WP2 is identical to the respective subsequence in WP\. (This subsequence includes those messages 
not involving the core variables, and includes messages of type x — > C and of the type C — > x.) 

Proof. First note that all messages x — > C, x € TC, do not change (sign) from the second iteration 
onwards (by the analysis in the proof of Proposition I24p . Furthermore, if £ x satisfies C in 99, then 
x — ► C is positive (if x is a true literal in C, or negative otherwise), and therefore all messages 
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C — > y, y 7^ x are constantly 0. Namely, they don't effect any calculation, and this is as if we 
replaced £ x with TRUE, and in the simplification process C disappeared. If £ x is false in C under 
<p, then x^Cis constantly negative (if £ x = x, or constantly positive if £ x = x), and this is exactly 
like having £ x removed from C (which is the result of the simplification process). 



7.3 Analysis of WP on the non-core factor graph 

Note that to prove the convergence of the algorithm we need also to prove that messages of the 
sort C — > x where C is not in the core and x is in the core converge. However, if we prove that all 
messages in the factor graph induced by the non-core variables converge, then this (with the fact 
that the core factor graph messages converge) immediately implies the convergence of messages of 
this type. Therefore, our goal reduces to proving convergence of WP on the factor graph induced by 
T\$, where ip assigns the core variables their planted assignment, and the rest are UNASSIGNED. 

We say that WP converged correctly in a connected component C of the non-core factor graph 
if there exists a satisfying assignment ip of the entire formula which is consistent with tp on the 
core, and with the assignment of WP to C. 

Consider a connected component in the non-core factor graph consisting of a cycle with trees 
hanging from it. Our analysis proceeds in three steps: 

(a) We first prove that clause-variable and variable-clause messages of the form a — > (3 where a — > 
(3 lead from the trees to the cycle, converge weakly correctly w.r.t. the planted assignment. 
In the case that the component has no cycles, this concludes the proof. 

(b) Then, using a refined case analysis, we show that the messages along the cycle also converge 
whp, this time not necessarily to the planted assignment, but to some satisfying assignment 
which agrees with the already converged messages. 

(c) Finally, we conclude by showing that messages from the cycles to the trees converge. More- 
over, this will imply that convergence will be to values consistent with the values converged to 
in step (a), and that hence that all messages in the connected component converge correctly 
according to some satisfying assignment. 

Consider the factor graph F induced by the simplified formula. A cycle in F is a collection 
xi, C2, X3, C4, . . . , x r = x\ where Xi and Xi + 2 belong to Cj+i for all i (in our description we consider 
only odd values of i) and Xi / Xj + 2, Cj + i / Cj+3 for all i. A factor graph F is a tree if it contains 
no cycles. It is unicyclic if it contains exactly one cycle. Let x — ► C be a directed edge of F. We 
say that x — > C belongs to the cycle, if both x and C belong to the cycle. For an edge x — > C that 
does not belong to the cycle, we say that x — > C is directed towards the cycle if x doesn't belong 
to the cycle and C lies on the simple path from x to the cycle. We say that the edge x — > C is 
directed away from the cycle if C doesn't belong to the cycle and x lies on the simple path from 
the cycle to C. Similarly we define what it means for an edges C — > x to belong to the cycle, to be 
directed towards the cycle and to be directed away from the cycle. 

Proposition 26. Let F be a unicyclic factor graph. Then every directed edge of the form x — > C 
or C — > x either belongs to the cycle, or is directed towards it or directed away from it. 
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Proof. Recall that the factor graph is an undirected graph, and the direction is associated with 
the messages. Take an edge x — ► C (similarly for C — > x), if it lies on the cycle, then we are done. 
Otherwise, since the factor graph is connected, consider the path in the tree leading from some 
element of the cycle to C. This path is either contained in the path to x or contains it (otherwise 
there is another cycle). In the first case x — > C is directed towards the cycle, and in the latter 
x — ► C is directed away from the cycle. ■ 

Our analysis proceeds in two parts: first we shall analyze WP on the trees, then WP on the 
cycle and connect the two (which is relevant for the uni-cyclic components). 

7.4 WP on the trees 

As we already mentioned before, there are two directions to consider: messages directed towards 
the cycle and away from the cycle. In this section we shall consider a rooted tree, and partition 
the messages according to messages which are oriented away from the root (they will correspond in 
the sequel to messages going away from the cycle) and messages towards the root (messages from 
the leaves towards the root - later to be identified with messages going into the cycle). The first 
lemma concerns messages going towards the root. 

Remark 27. Lemma [28] is a special case of the known fact (see [8] for example) that for every 
tree induced by a satisfiable formula, WP converges and there exists a satisfying assignment ip such 
that every Bi is either or agrees with ip. In Lemma [28] we assume that the formula is satisfiable 
by the all 1 assignment (the planted assignment), and consider only messages towards the root. 

Lemma 28. Let C — > x be an edge in the non-core factor graph belonging to a connected component 
of size s, and in particular to a rooted tree T . If C — > x is directed towards the root then the message 
C — > x converges after at most 0(s) iterations. Furthermore, if C — > x = 1 then x appears positively 
in C . 

Proof. We consider the case C = (£ x V £ y ) - the case C = (4VfjV4) where all three literals 
belong to non-core variables is proved similarly. For an edge (C, x) in the factor graph, we define 
level(C, x) to be the number of edges in a path between C and the leaf most distant from C in 
the factor graph from which the edge (C, x) is removed. The lemma is now proved using induction 
on the level i. Namely, after the i'th iteration, all messages C — > x associated with an edge (C,x) 
at level i converge, and if C — > x = 1 then x appears positively in C. 

The base case is an edge (C, x) at level 0. If level(C, x) = then C is a unit clause containing 
only the variable x. By the definition of the messages, in this case C — > x = 1 and indeed it must be 
the case that x is positive in C (as the other two variables evaluate to FALSE under the planted). 
Now consider an edge (C, x) at level i, and consider iteration i. Since i > 0, it must be that there 
is another non-core variable y in C (or two more variables y, z). Consider an edge (C',y), y £ C 
(if no such C exists that we are done as C — > x will be constantly in this case). 

level(C", y) is strictly smaller than i since every path from C to a leaf (when deleting the edge 
(C, x)) passes through some edge (C, y). By the induction hypothesis, all messages C — ► y already 
converged, and therefore also y — > C and in turn C — > x. It is only left to take care of the case 
C — > x = 1. In this case, there must be a clause C s.t. C' — > y = 1 and y appears positively in 
C (by the induction hypothesis). If C — > x = 1 it must be that y appears negatively in C and 
therefore x must appear positively (otherwise C is not satisfied by the planted assignment). ■ 
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Next we consider several scenarios that correspond to messages going form the root towards the 
leaves. Those scenarios correspond to step (c) of our analysis, referred to in Section [7.31 

Proposition 29. Assume that F is a unicyclic formula. Assume further that WP has converged 
on F. Let x — > C be directed away from the cycle. Let Fq be the subformula inducing the tree 
rooted at C , while x itself is removed from C . This formula contains all clauses whose path to the 
cycle goes via x and a clause corresponding to C where the literal corresponding to the variable x 
is removed (see Picl). Then C — > x = in the fixed point if and only if Fq is satisfiable. 

Proof. The structure of the proof is similar to that of Lemma [28j For convenience we extend the 
definition of level above as to include edges on the cycle. We say that an edge (C, x) in the factor 
graph has level(C, x) equal oo if (C, x) lies on a cycle and level(C, x) = t < oo if t is the maximal 
length of a path between C and a leaf in the factor graph from which the edge (C, x) is removed. 
The lemma is now proved using induction on t. 

The base case is an edge (C, x) at level 0. If level(C, x) = then C is a unit clause containing 
only the variable x, and then Fc is the empty formula. Indeed C — > x = 1 by definition and Fq is 
unsatisfiable (by definition again, the empty formula is not satisfiable). 

Now consider an edge (C, x) at level t > 0. Assume C = [x V i y V l z ) (maybe only iy). Observe 
that every edge (Cj,y) in Fq has level value which is strictly smaller than t - since every path 
from C to a leaf (when deleting the edge (C,x)) passes through some edge (Cj,y). First we prove 
that if Fq is not satisfiable then it must be that C — > x = 1. If Fc is not satisfiable then it must be 
that ly = y and l z = z (otherwise, if one of them is positive then ip satisfies Fc). Further, observe 
that there must exist at least one clause Cj containing y positively, and at least one C& containing 
z positively s.t. Fc { and Fc j are unsatisfiable. Otherwise, we can define ip' to be (p except that y or 
z are assigned FALSE (depending which of C% or Cj doesn't exist). It is easy to see that ip' satisfies 
Fc, contradicting our assumption. By the induction hypothesis C{ — > y = 1 and Cj — ► z = 1. 
This in turn implies that C — > X{ = 1 (since Cj and Cj contain y and z respectively in an opposite 
polarity to C and there cannot be any message Cj, — ► y = 1 where y appears negatively in Ck since 
Fc k is satisfiable in this case). Now assume that C — > x = 1. The same arguments imply that C 
must be of the form C = (x V y V z) and there exists Cj, Cj as above. By the induction hypothesis, 
if one is to satisfy Fc it must be that y = z = TRU E (this is the only way to satisfy Fc t and Fc } 
when inserting y back to Cj and z to Cj), but then C is not satisfied. ■ 

Proposition 30. Assume that F is a unicyclic formula. Assume further that WP has converged 
on F . Let C — > y be directed away from the cycle. Consider a subformula Fc which induces a tree 
rooted at a clause C . This formula contains the clause C and all other clauses whose path to the 
cycle goes via y. If in the fixed point for F it holds that 

• C^y = l, 

• y appears negatively in C , 

• y-C> 1, 

then WP converge correctly in Fc if (recall this means that there exists a satisfying assignment ip 
of the entire formula which is consistent with <p on the core, and with the assignment of WP to 
Fc). 
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Proof. Let us track the origin of the message y — ► C in the tree, (which is directed towards the 
root). For y — ► C > 1 to occur, there must be a clause D\ in the tree that contains y positively and 
a message D\ — > y = 1 in the direction of the root (as messages in the direction of the root are only 
effected by other messages in that direction). Let us backtrack one more step. D\ = (y V k V w) 
for some variables w, k; k and w must appear negatively in D\ by Lemma [28]), and the fact that 
D\ — > y = 1, that is both k and w were issued warnings having them not satisfy D±. Let us consider 
the clauses Z?2 and D% that issues warnings to k and w respectively. L>2 = {k V ...), D3 = (w V ...), 
L>2 — > A; = 1 and D3 — ► it; = 1, and both messages are directed towards the root. Obviously, one can 
inductively continue this backtracking procedure which terminates at the leaves of the tree (since 
there are not cycles the procedure is well defined and always terminates). Let us call the clauses 
and variables that emerge in this backtrack the spine of the tree. The figure below illustrates this 
procedure. 



a 



w 



k 



y 



C 



Figure 1: The spine of a tree 



For the subtree corresponding to the spinal variable we show that all messages that point away 
from the root converge in a consistent way with planted assignment. This combined with Lemma 
1281 completes that part of the proof. 

Let k be some variable that belongs to the spine, and let be the subformula corresponding 
to the tree hanging from k (and we think of the messages along that tree oriented away from the 
cycle). We claim that k — > C > for every C in F^. This is proven via induction on the distance 
of the variable from y. The base case is distance 0, which is y itself. The messages that we need 
to verify are of the form y — ► C, C ^ D\, which are pointing away from the cycle. In this case 
y — ► C > (y's message agrees with the planted); this is because the wrong message Cj+i — > y is 
evened by the correct warning D\ — > y, and y — > C' depends only on one message which is directed 
away from the cycle - otherwise there is a second cycle. The induction step follows very similarly. 
This fact guarantees correct convergence of the variables in the trees hanging from spinal variables. 
Finally, for the spinal variables, there is always at most one message in the direction away from the 
cycle (otherwise there is more than one cycle); this message may be wrong. Using a very similar 
inductive argument one can show that there is always at least one correct warning (in the direction 
of the cycle), therefore B w > for every spinal variable w. 

As for the non-spinal parts of the tree that hang on y, say a clause M 7^ D\, C, then y — > M > 
since the wrong message of C is evened by the correct warning of D\ (and there may be other 
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correct warnings from messages in the direction of the cycle) . Since there is a satisfying assignment 
of that subtree which is consistent with B y > 0, Remark [27] can be applied to guarantee correct 
convergence. ■ 



7.5 WP on cycles 

We will denote a cycle by xi, C2, X3, C±...xir-\i Ciri %i where by this we mean that Xi appears in 
the clauses before/after it and that Cj contains the two variables before/after it. We consider two 
different types of cycles. 

• Biased cycles: cycles that have at least one warning message C — > Xi = 1 coming into the 
cycle, where C — > Xi directs into the cycle and the value of C — > Xi is the value after the edge 
has converged. 

• Free cycles: cycles that do not have such messages coming in, or all messages coming in are 
messages. 

7.5.1 Convergence of WP when the cycle is biased: 

First we observe that we may assume w.l.o.g. that edges that enter the cycle enter it at a variable 
rather than at a clause (hence that every clause on the cycle contains exactly two non-core variables) . 
This is because of a simple argument similar to Proposition [25) consider an edge going into the 
cycle, z — > C, and w.l.o.g. assume that z appears positively in C. After all the edges going into 
the cycle have converged, if z — ► C > it follows that C — ► x = for cycle edges (C, x), and thus 
execution on the cycle is the same as if C was removed from the formula, only now we are left with 
a tree, for which convergence to a correct assignment is guaranteed (Remark I27p . If z — > C < 0, 
then the execution is exactly as if z was removed from C (and C is in 2-CNF form). 

Proposition 31. Let C be a connected component of the factor graph of size s containing one cycle 
s.t. there exists an edge directed into the cycle C — > Xi where Xi belongs to the cycle and such 
that the message converges to C — > X{ = 1. Then WP converges on C after at most O(s) rounds. 
Moreover for the fixed point, if the message C 1 — ► x = 1 then x appears positively in C . 

Proof. A message of the cycle Cj — ► Xj+i depends only on cycle messages of the type Cji — ► 
Xj/ + i,Xji + i — > Cji + 2 and on messages coming into the cycle. In other words during the execution 
of WP the values of all messages Cj> — > — > Cj/-2 do not effect the value of the message 
Cj — > Xj+i. Recall that we are in the case where there exists a message C — > x% = 1 going into the 
cycle (after the convergence of these messages). Also Xi must appear positively in C. We consider 
the following cases: 

• There exists a variable Xj that appears positively in both Cj-i and Cj+i (the case j = i is 
allowed here). We note that in this case the message Xj — ► Cj+i must take the value either 
or 1 which implies that the message C,-+i — > converges to the value 0. This in turn 
implies that the value of all messages x r — > C r +i and CV+i — ► x r +2 f° r r 7^ j wm remain the 
same if the clause Cj + i is removed from the formula. However, this case reduces to the case 
of tree formula. 
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• Xi appears negatively in Cj+i and positively in Cj_i. We note that in this case the message 
Xi — > Ci + i always take the value 1 which implies that the message Cj+i — ► Xj+2 always take 
the value 1. Thus in this case we may remove the clause Cj+i from the formula and replace it 
by the unit clause l y where Cj+i = l y \l X{. Again, this reduces to the case of a tree formula. 

• The remaining case is the case where x% appears negatively in both Cj_i and Cj+i and there 
is no j such that Xj appears positively in both Cj-i and Cj+\. We claim that this leads 
to contradiction. Note that by the lemma above there exists a satisfying assignment where 
Xi = 1. Write Cj+i = xi V £i+2- Then for the truth assignment we must have £i + 2 = 1, 
similarly £i + 4 = 1 etc. until we reach x~i = 1 - a contradiction. 

To summarize, by Lemma[28]the messages going into the rooted tree at Xi converge after O(s) steps, 
and at least one warning is issued. By the above discussion, for every clause D in the connected 
component it holds that Xi — > D > (as X{ appears in at most one message which may be wrong - 
a cycle message). Since there is always a satisfying assignment consistent with Xi assigned TRUE, 
then after reducing the cycle to a tree we are left with a satisfiable tree. Remark [27] guarantees 
convergence in additional O(s) iterations. ■ 



7.6 Convergence of WP when the cycle is free 

The main result of this subsection is summarized in the following claim: 

Proposition 32. Let C be a connected component of the factor graph of size s containing one cycle 
of size r s.t. the fixed point contains no messages C — > x = 1 going into the cycle (the cycle is 
free). Then whp WP converges on C after at most 0(r 2 • logn + s) rounds. Moreover for the fixed 
point, if we simplify the formula which induces C according to the resulting Bi 's, then the resulting 
subformula is satisfiable. 

Remark 33. Observe that the free case is the only one where convergence according to the planted 
assignment is not guaranteed. Furthermore, the free cycle case is the one that may not converge 
"quickly" (or not at all), though this is extremely unlikely. The proof of Proposition [32] is the only 
place in the analysis where we use the fact that in line 3 . a of WP we use fresh randomness in every 
iteration. 

We consider two cases: the easy one is the case in which the cycle contains a pure variable w.r.t 
the cycle (though this variable may not be pure w.r.t to the entire formula). 

Proposition 34. If the cycle contains a variable x% appearing in the same polarity in both Cj+i, Cj_i, 
then the messages C — ► x along cycle edges converge. Moreover for the fixed point, if C — * x = 1 
then x satisfies C according to ip. 

The proof is very similar to the first case in the proof of Proposition [31] We omit the details. 
We now move to the harder case, in which the cycle contains no pure variables (which is the case 
referred to in Remark 133 1> . 

Proposition 35. Consider a free cycle of size r with no pure literal, and one of the two directed 
cycles of messages. Then the messages along the cycle converge whp to either all or all 1 in 
0(r 2 log n) rounds. 
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Convergence whp in polynomial time suffices due to Corollary [15] (which asserts that whp 
every cycle is of length at most log £ n for every e > 0). The proof of Proposition [35] is given in the 
end of this section. We proceed by analyzing WP assuming that Proposition 1351 holds, which is the 
case whp. 

Proposition 36. Suppose that the cycle messages have converged (in the setting of Proposition [35\) , 
then the formula resulting from substituting every x^ with the value assigned to it by Bi ( according 
to the fixed point of WP), and simplifying, is satisfiable. 

Proof. Let F be the subformula that induces the connected component C, and decompose it 
according to the trees that hang on the cycle's variables and the trees that hang on the cycle's 
clauses. Observe that the formulas that induce these trees are variable and clause disjoint (since 
there is only one cycle in the C). 

Let us start with the cycle clauses. The key observation is that setting the cycle variables 
according to one arbitrary orientation (say, set Xi to satisfy Q+i) satisfies the cycle and doesn't 
conflict with any satisfying assignment of the hanging trees: if the tree hangs on a variable Xj, 
then since the cycle is free, the tree is satisfiable regardless of the assignment of x, (Proposition 
l29j) . In the case that the tree hangs on a cycle-clause C, then the cycle variables and the tree 
variables are disjoint, and C is satisfied already by a cycle-variable regardless of the assignment of 
the tree- variables. Now how does this coincide with the result of WP. Recall that we are in the 
case where the cycle is free. Therefore only messages C — > Xi where both C and Xi belong to the 
cycle effect Bi. If in the fixed point one cycle orientation is and one orientation is 1, then the Bi 
messages of the cycle variables implement exactly this policy. If both cycle orientations converged 
to 1 or to 0, then the corresponding Bi messages of all cycle variables are UNASSIGNED (since the 
cycle is free), but then the same policy can be used to satisfy the clauses of the cycle in a manner 
consistent with the rest of the formula. 

It remains to show that WP converges on every tree in a manner that is consistent with some 
satisfying assignment of the tree. We consider several cases. 

Consider a tree hanging on a cycle variable Xi- Let C be some non-cycle clause that contains 
Xi, and Fq the subformula that induces the tree rooted at C. Observe that once the cycle has 
converged, then the message x% — > C does not change anymore. If Xj — ► C agrees with tp there are 
two possibilities. Either X{ satisfies C under ip, in which case C always sends to Fc, and then 
WP executes on Fc as if C is removed. Remark [27] guarantees correct convergence (as Fq \ C is 
satisfiable), and as for C, Bi > and we can set Xi to TRUE so that it satisfies C and is consistent 
with the assignment of the cycle (Bi > since X{ > and C — > X, = as we are in the free cycle 
case). If Xi appears negatively in C, then WP executes as if X{ was deleted from C. Still Fc is 
satisfiable and correct convergence is guaranteed. 

Now consider the case where X, — > C disagrees with ip. Recall that we assume p(xi) = TRUE, 
and therefore x — > C is negative in the fixed point. If Xi appears negatively in C then C — > y = 
for every y G C (since Xj signals C that it satisfies it), and therefore C doesn't effect any calculation 
from this point onwards, and the correct convergence of Fc is again guaranteed by Remark 1271 on 
the convergence for satisfiable trees. The more intricate case is if C contains Xi positively. Since 
we are in the free case, it must hold that C — > x = 0. Therefore using Proposition [29] one obtains 
that Fc is satisfiable (regardless of the assignment of Xi), and WP will converge as required (again 
Remark [27]). 

Now consider a tree hanging on a cycle clause. Namely, Cj+i = (xi V V y), where Xi,Xi + 2 
are cycle variables, and (Ci + \,y) is a tree edge. If one of the cycle orientations converged to 0, 
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then Cj_|_i — > y converges to 0, and then Remark [27] guarantees correct convergence. The same 
applies to the case where C^+i — ► V converges to 1 and y is positive in Cj+i (since then we can 
remove y from the tree, use Remark[27]for the remaining part, then add back y, and set it to TRUE 
without causing any conflict with the tree assignment, but satisfying Cj+i according to the planted 
assignment). 

The delicate case remains when Cj+i — ► y converges to 1 but y's polarity in Cj+i disagrees 
with if, that is, y is negative in Ci + \. The key observation is that the message y — > Cj+i (which 
is directed towards the cycle) must have converged to a positive value (otherwise, Cj+i — ► X{ and 
Cj+i — > Xj + 2 would have converged to 0). However this complies with the scenario of Proposition 
[301 and again correct convergence is guaranteed. ■ 

In Theorem [1] the unassigned variables are required to induce a "simple" formula, which is 
satisfiable in linear time. Observe that the factor graph induced by the UNASSIGNED variables 
consists of connected components whose structure is a cycle with trees hanging on it, or just a tree. 
A formula whose factor graph is a tree can be satisfied in linear time by starting with the leaves 
(which are determined uniquely in case that the leaf is a clause - namely, a unit clause, or if the leaf 
is a variable then it appears only in one clause, and can be immediately assigned) and proceeding 
recursively. Regarding the cycle, consider an arbitrary variable x on the cycle. By assigning x and 
simplifying accordingly, we remain with a tree. Since there are only two ways to assign x, the whole 
procedures is linear in the size of the connected component. This completes the proof of Theorem 

m 



7.7 Proof of Proposition 

Since the cycle has no pure literal it must be of the following form: C\ = (£ Xl V £ X2 )iC2 = 
(^X2 V ^£3)1 • • • 1 Cl = (^x L V 

Consider one of the directed cycles, say: x\ — > C\ — > X2 — > ■ • • and note that when the message 
Xi — ► d is updated it obtains the current value of C%-\ — ► xi and when the message Cj — ► Xi+i is 
updated, it obtains the current value of Xi — ► Cj. 

It thus suffices to show that the process above converges to all or all 1 in time polynomial in 
the cycle length. This we prove in the lemma below. 

Proposition 37. Consider the following process on {0, 1} L . Given the state of the process 
at round i, the state Y l+l at round i + 1 is defined by choosing a permutation a € Sk uniformly 
at random. Then let Aq = T l and for 1 < j < L, let A,- be obtained from Aj_i by setting 
Aj(a(j — 1)) = Aj_i((o"(j — 1) + 1) mod L) and Aj(i) = A,_i(i) for all i 7^ o~(j — 1). Finally, let 
L ■ 

Let T be the stopping time where the process hits the state all or all 1. Then 



T i+1 = A 



Pr[T > 4aL 2 } < L2~ a . (7.1) 

for all a > 1 integer. 

The proof of Proposition [37] is based on the fact that the process defined in this lemma is a 
martingle. This is established in the following two lemmas. 



Lemma 38. Consider the following variant F l of the process F l defined in Proposition 37. In the 
variant, different intervals of 0/1 are assigned different colors and the copying procedure is as above. 
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Fix one color and let denote the number of elements of the cycle of that color in T l . Then Xi 
is a martingle with respect to the filtration defined by T l . 

Proof. From the definition of the copying process it is easy to see that 

E[Xi + i\f l ,t l 1 ,...]= E[Xi + i\Xi]. 

We will show below that = and thus that Xi is a martingle with respect to the 

filtration T\ 

Assume that Xi = k. Then w.l.o.g. we may assume that the configuration T l consists of an 
interval of l's of length k and an interval of O's of length L — k. We calculate separately the expected 
shifts in the locations of left end-points of the and 1 interval respectively. We denote the two shift 
random variables by Lq and L\. Clearly Lq = /o,i + Io,2 + • • • + Io,k-i where Lqj is the indicator of 
the event that left-end point shifted by at least j and similarly for L\. Note that 

E[Ioj] = Jl ~ (L-l+j)\ 

and that 

E[Il ' j] = J~ Tkh)v 

The last equation follows from the fact that in order for the 1 interval to extend by at least j, the 
j copying has to take place in the correct order and it is forbidden that they all took place in the 
right order and the interval has become a interval. The previous equation is dervied similarly. 
Thus 

Em+1 - *,)|*,, . em - em - g (I - ^) - ± (I - Jr -^) . 

This concludes the proof that Xi is a martingle. The proof follows. ■ 



The proof of Proposition [37] follows by a union bound from the following lemma where the union 
is taken over all intervals. 

Lemma 39. Consider the process T l defined in Lemma \38[ Fix one interval and let Xi denote its 
length. Let T be the stopping time where Xi equals either or L. Then 

P[T > AaL 2 ] < 2' a . 

Proof. In order to bound the hitting probability of and L, we need some bounds on the variance 
of the martingale differences. In particular, we claim that unless k = or k = L it holds that 

E[(X t+l -X t f\X t = k) > 1/2. 

Iik = loik = L— 1 this follows since with probability at least 1/2 the end of the interval will be 
hit. Otherwise, it is easy to see that the probability that Xt+i — Xt is at least 1 is at least 1/4 and 
similarly the probability that it is at most —1 is at least 1/4. This can be verified by considering 
the event that one end-points moves by at least 2 and the other one by at most 1. 
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Let T be the stopping time when Xj- hits or n. Then by a Wald kind of calculation we obtain: 

oo 

L 2 > E[{X T -X ) 2 \=E[{Y J l{T>t)(X t -X t ^ 1 )) 2 \ 

t=i 

oo 

= E[J2(Xt ~ X t ^)(X s -X.-i)l(T > maxM)] 
t,s=l 

= E[J2(Xt ~ X t ^) 2 l{T >t)]>-J2 P[T >t] = E[T]/2, 
t=i t=i 

where the first equality in the last line follows from the fact that if s < t say then: 

E[(X t - X t -!)(X a -X S _!)1(T > maxi,s)] 

= E[(X t - X t _i)(X, - X s _i)(l - 1(T < i))] 

= - X t _i)(X, - - 1(T < . . . ,X t _i]] 

= - X s _i)(l - 1(T < i))£[X t " *t-i|*i, • • • ,Xt-i]] = 0. 

We thus obtain that E[T] < 2L 2 . This implies in turn that P[T > 4L 2 ] < 1/2 and that 
P[T > AaL 2 ] < 2~ a for a > 1 since Xt is a Markov chain. The proposition follows. 



8 Discussion 

Our results show that WP is effective in solving Vn]p nt - Though not being the first to give an 
algorithm for Vn]p nt , our results are a first example of rigourously analyzing a message passing 
algorithm on a natural non-trivial random SAT distribution. We remark that our goal was to 
analyze WP under its most common definition, resisting attempts to modify the algorithm in ways 
that would simplify the analysis. One such simplification would result if in the first few iterations, 
messages are updated in parallel in two phases: clause- variable messages updates and then variable- 
clause messages updates. 

In the non-planted case, for low density formulas (considerably below the satisfiability thresh- 
old) , some algorithms were rigorously shown to find whp a satisfying assignment efficiently [21 [9] . 
Experimental results predict that as the density of the formula increases, more sophisticated al- 
gorithms are needed in order to find a satisfying assignment. At higher densities (closer to the 
satisfiability threshold), there is a major gap between the experimental performance of the best 
known algorithms [8] (a message passing algorithm that experimentally works at density ~4.2), 
and the best rigorously-analyzed algorithm [23] (density 3.52). 

One possible explanation for the increasing computational hardness of finding solutions in the 
non-planted case is based on the geometry of the space of satisfying assignment. It is now established 
[281 [1] that for A:-SAT just below the satisfiability threshold and k > 8, the space of solutions 
decomposes into an exponential number of (Hamming-distance) connected clusters such that the 
distance between each two is linear in the number of variables. Such complex geometry of the space 
of solutions poses a complex algorithmic challenge. 
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Our results, and similarly |16l 115] , show that in the planted case (with density some large con- 
stant above the satisfiability threshold), the algorithmic task of finding a satisfying assignment is 
relatively easy, and in particular, the naive WP algorithm is effective in finding satisfying assign- 
ments. Planted formulas in this regime have only one cluster of satisfying assignments (see jlOj for 
more discussion). 

We conclude with an open problem. Can our analysis be extended to show that Belief Propaga- 
tion (BP) finds a satisfying assignment to "Pn|p nt in the setting of Theorem [Tp Experimental results 
predict the answer to be positive. However, our analysis of WP does not extend as is to BP. In 
WP, all warnings received by a variable (or by a clause) have equal weight, but in BP this need 
not be the case (there is a probability level associated with each warning). In particular, this may 
lead to the case that messages received from non-core portions of the formula can effect the core, 
a possibility that our analysis managed to exclude for the WP algorithm. 
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